The Past, Present, and Future(s): Verifying Temporal Software Properties

Software systems are increasingly present in every aspect of our society, as their deployment can be witnessed from seemingly trivial applications of light switches, to critical control systems of nuclear facilities. In the context of critical systems, software faults and errors could potentially lead to detrimental consequences, thus more rigorous methodologies beyond the scope of testing need be applied to software systems. Formal verification, the concept of being able to mathematically prove the correctness of an algorithm with respect to a mathematical formal specification, can indeed help us prevent these failures. A popular specification language for these formal specifications is temporal logic, due to its intuitive, yet precise expressions that can be utilized to both specify and verify fundamental properties pertaining to software systems. Temporal logic can express properties pertaining to safety, liveness, termination, non-termination, and more with regards to various systems such as Windows device drivers, kernel APIs, database servers, etc. This dissertation thus presents automated scalable techniques for verifying expressive temporal logic properties of software systems, specifically those beyond the scope of existing techniques. Furthermore, this work considers the temporal sub-logics fair-CTL, CTL*, and CTL*lp, as verifying these more expressive sub-logics has been an outstanding research problem. We begin building our framework by introducing a novel scalable and high-performance CTL verification technique. Our CTL methodology is unique relative to existing techniques in that it facilitates reasoning about more expressive temporal logics. In particular, it allows us to further introduce various methodologies that allow us to verify fair-CTL, CTL*, and CTL*lp. We support the verification of fair-CTL through a reduction to our CTL model checking technique via the use of infinite non-deterministic branching to symbolically partition fair from unfair executions. For CTL∗, we propose a method that uses an internal encoding which facilitates reasoning about the subtle interplay between the nesting of path and state temporal operators that occurs within CTL∗ proofs. A precondition synthesis strategy is then used over a program transformation which trades nondeterminism in the transition relation for nondeterminism explicit in variables predicting future outcomes when necessary. Finally, we propose a linear-past extension to CTL*, that being CTL*lp, in which the past is linear and each moment in time has a unique past. We support this extension through the use of history variables over our CTL∗ technique. We demonstrate the fully automated implementation of our techniques, and report our bench- marks carried out on code fragments from the PostgreSQL database server, Apache web server, Windows OS kernel, as well as smaller programs demonstrating the expressiveness of fair-CTL, CTL*, and CTL*lp specifications. Together, these novel methodologies lead to a new class of fully automated tools capable of proving crucial properties that no tool could previously prove in the infinite-state setting

Verifying Increasingly Expressive Temporal Logics for Infinite-State Systems

Temporal logic is a formal system for specifying and reasoning about propositions qualified in terms of time. It offers a unified approach to program verification as it applies to both sequential and parallel programs and provides a uniform framework for describing a system at any level of abstraction. Thus a number of automated systems have been proposed to exclusively reason about either Computation-Tree Logic (CTL) or Linear Temporal Logic (LTL) in the infinite-state setting. Unfortunately, these logics have significantly reduced expressiveness as they restrict the interplay between temporal operators and path quantifiers, thus disallowing the expression of many practical properties, for example “along some future an event occurs infinitely often”. Contrarily, CTL∗, a superset of both CTL and LTL, can facilitate the interplay between path-based and state-based reasoning. CTL∗ thus exclusively allows for the expressiveness of properties involving existential system stabilization and “possibility” properties. Until now, there have not existed automated systems that allow for the verification of such expressive CTL∗ properties over infinite-state systems. This paper proposes a method capable of such a task, thus introducing the first known fully automated tool for symbolically proving CTL∗ properties of (infinite-state) integer programs. The method uses an internal encoding that admits reasoning about the subtle interplay between the nesting of temporal operators and path quantifiers that occurs within CTL∗ proofs. A program transformation is first employed that trades nondeterminism in the transition relation for nondeterminism explicit in variables predicting future outcomes when necessary. We then synthesize and quantify preconditions over the transformed program that represent program states that satisfy a CTL∗ formula. This paper demonstrates the viability of our approach in practice, thus leading to a new class of fullyautomated tools capable of proving crucial properties that no tool could previously prove. Additionally, we consider the linear-past extension to CTL∗ for infinite-state systems in which the past is linear and each moment in time has a unique past. We discuss the practice of this extension and how it is further supported through the use of history variables. We have implemented our approach and report our benchmarks carried out on case studies ranging from smaller programs to demonstrate the expressiveness of CTL∗ specifications, to larger code bases drawn from device drivers and various industrial example

Faster temporal reasoning for infinite-state programs

In many model checking tools that support temporal logic, performance is hindered by redundant reasoning performed in the presence of nested temporal operators. In particular, tools supporting the state-based temporal logic CTL often symbolically partition the system's state space using the sub-formulae of the input temporal formula. This can lead to repeated work when tools are applied to infinite-state programs, as often the characterization of the state-spaces for nearby program locations are similar and interrelated. In this paper, we describe a new symbolic procedure for CTL verification of infinite-state programs. Our procedure uses the structure of the program's control-flow graph in combination with the nesting of temporal operators in order to optimize reasoning performed during symbolic model checking. An experimental evaluation against competing tools demonstrates that our approach not only gains orders-of-magnitude performance speed improvement, but allows for scalability of temporal reasoning for larger programs

Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims

With the recent wave of progress in artificial intelligence (AI) has come a growing awareness of the large-scale impacts of AI systems, and recognition that existing regulations and norms in industry and academia are insufficient to ensure responsible AI development. In order for AI developers to earn trust from system users, customers, civil society, governments, and other stakeholders that they are building AI responsibly, they will need to make verifiable claims to which they can be held accountable. Those outside of a given organization also need effective means of scrutinizing such claims. This report suggests various steps that different stakeholders can take to improve the verifiability of claims made about AI systems and their associated development processes, with a focus on providing evidence about the safety, security, fairness, and privacy protection of AI systems. We analyze ten mechanisms for this purpose--spanning institutions, software, and hardware--and make recommendations aimed at implementing, exploring, or improving those mechanisms